Skip to content

FIX: don't create 32 bit int type for int64 column (GH7433) #7634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 5, 2014

Conversation

jorisvandenbossche
Copy link
Member

Closes #7433

Replace the sqlalchemy Integer type with BigInteger as the default integer type in pandas is int64.
Probably we could be more specific in the type conversion (check for the exact integer type).

@jorisvandenbossche jorisvandenbossche added this to the 0.14.1 milestone Jul 1, 2014
@jreback jreback added the Dtypes label Jul 1, 2014
@jorisvandenbossche
Copy link
Member Author

Tests are passing. @mangecoeur @danielballan OK with this?

@danielballan
Copy link
Contributor

Looks good.

@jorisvandenbossche
Copy link
Member Author

Could someone test it with oracle?

jorisvandenbossche added a commit that referenced this pull request Jul 5, 2014
FIX: don't create 32 bit int type for int64 column (GH7433)
@jorisvandenbossche jorisvandenbossche merged commit f6a5f41 into pandas-dev:master Jul 5, 2014
@jorisvandenbossche jorisvandenbossche deleted the sql-int64 branch July 5, 2014 13:48
@jorisvandenbossche
Copy link
Member Author

I see that there are on travis a lot of warnings after merging this. So the tests pass, but raise warnings like: /home/travis/virtualenv/python2.6.9/lib/python2.6/site-packages/sqlalchemy/engine/reflection.py:46: SAWarning: Did not recognize type 'BIGINT' of column 'index'

I didn't see this warning, and after testing, I don't see it with sqlalchemy 0.9.6 and 0.8.3, but I do see it with 0.8.1 (the version used on travis for build 3).

As the tests pass, I assume this is not a problem. But anybody an idea to prevent this warning is raised when users have an older version of sqlalchemy?

@jreback
Copy link
Contributor

jreback commented Jul 6, 2014

so u want to put in place one/both of those newer versions on Travis? which ones should I replace?

@jorisvandenbossche
Copy link
Member Author

Well, already different version are used on different builds on travis. The older version 0.7.1 is used on py2.6, 0.8.1 is used on py2.7 and 0.9.1 is used on py3.3. So the older version of 0.7.1 should be kept to test compatibility with older sqlalchemy, the other ones could be updated to the latest sqlalchemy version.

@jorisvandenbossche
Copy link
Member Author

This issue was solved in 0.8.2 (https://bitbucket.org/zzzeek/sqlalchemy/issue/2764/make-sqlite-handle-bigint).

So, or we live with the fact that users with older versions of sqlalchemy get this warning, or we can also try to 'fix' it ourselves with something like: https://review.openstack.org/#/c/23883/4/nova/db/sqlalchemy/__init__.py (from, a similar nova bug)

@jreback
Copy link
Contributor

jreback commented Jul 6, 2014

I wouldn't worry about in that case; changing the 3.4 build to use 0.9.6

@jorisvandenbossche
Copy link
Member Author

You can also change the 2.7 build to use 0.9.6, then the warning don't noise the travis test output (and the older 0.7.1 is still tested in 2.6)

@jreback
Copy link
Contributor

jreback commented Jul 6, 2014

both done (Travis still building master though)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dtype Conversions Unexpected or buggy dtype conversions IO SQL to_sql, read_sql, read_sql_query
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SQLAlchemy MySQL df.to_sql() creates 32 bit int MySQL columns for int64 df cols
4 participants